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CONTRACTUAL  ITEMS 


The  following  items  are  specifically  requested  in  the  contract,  and  are  addressed  below: 

1.  Accomplishments  compared  to  goal  objectives-,  the  accomplishments  far  exceed  the  original  goal,  which  was 
to  create  an  XML  “presence”  for  use  in  the  battle  infosphere.  We  not  only  created  two  XML 
languages,  schemas,  and  implementations  around  them,  we  also  have  extended  this  work  to  ontology 
definitions  and  use  in  the  user  interface.  We  constructed  a  comprehensive  3D  dynamic  model 
authoring  environment,  linking  dynamic  model  with  geometric  model  components. 

2.  Reasons  why  established  goals  were  not  met  if  appropriate.  N/ A. 

3.  Cumulative  list  of  publications-,  see  the  Results  <&  Discussion  section. 

4.  List  of  professional personnel.  Paul  Fishwick  (PI)  and  students.  Students  are  listed  toward  the  end  of  this 
report.  Requested  info  for  the  PI  is  PhD  Computer  and  Information  Science,  University  of 
Pennsylvania,  1986.  Thesis  title:  “Hierarchical  Reasoning:  Simulating  Complex  Processes  over 
Multiple  Levels  of  Abstraction”. 

5.  Papers  presented  at  meetings-.  See  the  comprehensive  list  under  Results  <&  Discussion. 

6.  Consultive  and  advisory  functions-.  N/A. 

7.  Inventions  and  Patents-.  The  software  that  we  have  constructed  (SimPackJ,  SimPackP,  and  RUBE)  is 
open  source. 


INTRODUCTION 

For  the  past  four  years,  we  have  developed  a  software  framework  called  RUBE.  RUBE’s  purpose  is  to 
provide  a  modeler  with  a  way  in  which  to  better  integrate  the  phenomenon  being  modeled  and  the  model 
itself.  This  integration  is  done  using  multiple  visual  modes  of  display,  allowing  the  dynamic  models,  as  well 
as  the  phenomena,  to  be  displayed  in  3D.  Figure  1  displays  the  RUBE  architecture.  Our  use  of  ontologies 
within  the  RUBE  project  is  founded  on  two  approaches:  (1)  schema  definitions  and  XML  files  for  model 
types  and  model  files;  and  (2)  an  OWL  representation  of  a  sample  air  reconnaissance  scene.  We  proceed 
with  these  in  sequence. 

RUBE  begins  its  process  with  two  types  of  interfaces:  a  2D  interface  using  the  SodiPodi  tool,  and  a  3D  one 
using  Blender,  which  is  a  tool  for  authoring  and  animating  3D  scenes.  The  simulation  analyst  builds  a  scene  to 
be  simulated,  and  then  builds  dynamic  models  of  that  scene.  The  dynamic  models  are  translated  into  MXL 
(Multimodel  eXchange  Language).  MXL  contains  an  ontology  (or  XML  schema)  defining  certain  model  types 
and  how  they  are  defined.  For  example,  a  Finite  State  Machine  contains  an  initial  state,  a  set  of  states,  and 
transitions.  DXL  (Dynamics  eXchange  Language)  is  a  lower-level  homogeneous  block-model  language  capable 
of  describing  both  synchronous  and  asynchronous  execution  of  block  networks.  As  such,  DXL  networks 
reflect  behaviors  such  as  those  found  in  digital  circuits  as  well  as  more  loosely  connected  data  flow  networks. 
Both  MXL  and  DXL  are  XML  languages.  Each  has  a  schema,  defining  the  language  as  an  ontology. 
Returning  to  the  process  defined  in  Figure  1,  a  model  is  converted  into  MXL  and  then  DXL,  and  finally  into 
a  target  language  such  as  Java  or  Javascript.  This  Javascript  code  is  then  reinserted  into  the  scene.  This  is 
done  by  first  exporting  the  Blender  scene  into  an  X3D  (extensible  3D)  file  and  then  defining  the  Javascript 
in  X3D  script  nodes.  The  final  X3D  scene  file,  intact  with  both  geometric  and  dynamic  properties,  is  then 
executed  to  yield  the  simulation.  In  addition  to  the  work  performed  in  RUBE,  we  have  recently  started  to 
create  an  ontology  that  attempts  to  bring  all  knowledge  about  an  application  domain  together.  Figure  2 
shows  a  network  relating  elements  of  a  scene  (JSTARS,  FI  5,  and  UAV)  with  the  geometry  and  dynamics  of 
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these  objects  we  have  listed  the  defined  styles  that  might  be  required  during  the  formatting  process.  The 
idea  here  is  that  we  can  use  RUBE  to  generate  objects,  that  are  then  updated  in  the  ontology.  Moreover,  we 
can  add  to  the  ontology  manually,  if  necessary,  to  express  the  semantic  relations.  We  are  using  OWL  for 
expressing  the  network,  and  the  Stanford  Protege  tool  for  managing  the  links.  There  is  a  two-way 
connection  between  Blender  and  Protege  so  that  information  can  be  entered  in  Protege  and  then  appear  in 
Blender,  and  vice  versa. 


Figure  1:  RUBE  Framework,  from  Interface  to  Code  Generation 
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Figure  2:  An  Ontology  Defining  an  Air  Battle  Scene  with  Reconnaissance  and  Surveillance 

Aircraft 
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RESULTS  &  DISCUSSION 


This  contract  has  resulted  in  a  comprehensive  set  of  published  results  which  are  defined  below.  We 
list  the  books  completed,  archival  journal  articles,  and  conference  articles  published.  The  contract 
also  permitted  three  students  to  earn  their  doctoral  degrees. 


Multimedia  Collections 

•  Paul  A.  Fishwick,  Dynamic  Systems  Modeling,  3  CDs  overviewing  the  nature  of  dynamic 
models,  and  why  they  are  created. 


Books 

•  Paul  A.  Fishwick  and  Benjamin  Lok." Proceedings  of  the  2005  Conference  on  Human- 
Computer  Interface  Advances  for  Modeling  and  Simulation  (SIMCHI  ’05)",  Society  for 
Computer  Modeling  and  Simulation  International,  2005.  84pp. 

•  Paul  A.  Fishwick.  '  'Aesthetic  Computing"  MIT  Press,  sent  to  publisher,  expected 
publication  date  -  Fall  2005. 


Archival  Journal  Articles 

•  John  Hopkins  and  Paul  A.  Fishwick.  "Synthetic  Human  Agents  for  Modeling  and 
Simulation",  Proceedings  of  the  IEEE  in  special  issue  '  'Agent-Based  Modeling  and 
Simulation:  Exploiting  the  Metaphor",  February  2001,  Volume  89,  Number  2,  pp.  131-147. 

•  John  A.  Miller,  Paul  A.  Fishwick,  Simon  J.E.  Taylor,  Perakath  Benjamin  and  Boleslaw 
Szymanski,  "Research  and  Commercial  Opportunities  in  Web-Based  Simulation"  Simulation 
Practice  and  Theory  (SPT),  Special  Issue  on  Web-Based  Simulation,  Vol.  9,  No.  1-2 
(October  2001),  pp.  55-72,  Elsevier  Science. 

•  Paul  A.  Fishwick.  "Aesthetic  Programming:  Crafting  Personalized  Software",  Leonardo, 
MIT  Press,  2002,  Volume  35,  Number  4,  pp.  383-390,  2002. 

•  Paul  A.  Fishwick.  "Aesthetic  Computing:  Making  Artistic  Mathematics  and  Software", 
Special  Issue  on  Art  and  Programming,  YLEM  Journal  (Artists  Using  Science  & 
Technology),  10  (22),  September  -  October  2002,  pp.  6-11. 

•  John  Hopkins  and  Paul  A.  Fishwick,  '  'Exploiting  an  Agent-Based  Metaphor  in  Software 
Visualization  using  the  rube  System",  Journal  of  Visual  Languages  and  Computing,  14  (1), 
Feb.  2003,  pp.  97-117. 
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•  Taewoo  Kim,  Jinho  Lee,  and  Paul  Fishwick.  "A  Two-Stage  Modeling  and  Simulation 
Process  for  Web-Based  Modeling  and  Simulation",  ACM  Transactions  on  Modeling  and 
Computer  Simulation,  12  (3),  July  2002,  pp.  230-248. 

•  Paul  Fishwick.  ''Aesthetic  Computing  Manifesto",  Leonardo,  MIT  Press,  36  (4),  August 
2003,  pp.  255-256. 

•  Paul  A.  Fishwick.  "Toward  an  Integrative  Multimodeling  Interface:  A  Human-Computer 
Interface  Approach  to  Interrelating  Model  Structures",  in  the  "Grand  Challenges"  special 
issue  of  SCS  Transactions  on  Modeling  and  Simulation,  80  (9):  421-432,  September  2004. 

•  Hyunju  Shim  and  Paul  Fishwick.  '  'RUBE2D:  A  Web-Based  Customizable  2D  Modeling  and 
Simulation  System",  International  Journal  of  Simulation  and  Process  Modeling  (IJSPM),  1:  2- 
15.  January  2004. 

•  Paul  A.  Fishwick,  Stephan  Diehl,  Jane  Prophet  and  Jonas  Lowgren.  "Perspectives  in 
Aesthetic  Computing",  Leonardo  38  (2):  133-141,  MIT  Press,  2005. 

•  Paul  Fishwick.  "Nurturing  Next-Generation  Computer  Scientists",  IEEE  Computer, 
December  2003,  pp.  106-108. 

•  Paul  Fishwick,  Timothy  Davis,  and  Jane  Douglas.  "An  Empirical  Study  of  Aesthetic 
Computing",  submitted  to  SCS  Transactions  on  Modeling  and  Simulation,  April  2005. 

•  Paul  Fishwick.  "Enhancing  Experiential  and  Subjective  Qualities  of  Discrete  Structure 
representations  with  Aesthetic  Computing",  accepted  for  Journal  of  Visual  Languages  and 
Computing  (JVLC),  May  2004. 

•  John  A.  Miller,  Gregory  T.  Baramidze,  Amit  P.  Sheth,  and  Paul  A.  Fishwick  "Ontologies  for 
Modeling  and  Simulation:  Initial  Framework",  submitted  to  ACM  Transactions  on  Modeling 
and  Computer  Simulation,  October  2004. 

•  Minho  Park  and  Paul  Fishwick.  "Ontology-based  Customizable  3D  Modeling  for 
Simulation"  submitted  to  SCS  Transactions  on  Modeling  and  Simulation,  February  2005. 


Refereed  Conference  Articles 

•  Davis,  Paul;  Fishwick,  Paul;  Overstreet,  Michael;  Pegden,  Dennis,  '  'Model  Composability  as 
a  Research  Investment"  In  Proceedings  of  the  2000  Winter  Simulation  Conference,  Orlando, 
Florida,  December  10-13,  2000.  pp.  1585-1591. 

•  Fishwick,  Paul.  "On  Web-Based  Models  and  Repositories",  In  Proceedings  of  Enabling 
Technology  for  Simulation  Science,  Part  of  SPIE  AeroSense  '01  Conference,  Orlando, 
Florida,  April  16-18.  pp.  11-16. 
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•  Dance,  Linda  and  Fishwick,  Paul.  '  'A  Methodology  for  the  3D  Modeling  and  Visualization 
of  Concurrency  Networks",  In  Proceedings  of  Enabling  Technology  for  Simulation  Science, 
Part  of  SPIE  AeroSense  ’01  Conference,  Orlando,  Florida,  April  16-18.  pp.  152-163. 

•  Kim,  Taewoo  and  Fishwick,  Paul.  ''Virtual  Reality  Modeling  Language  Templates  for 
Dynamic  Model  Construction",  In  Proceedings  of  Enabling  Technology  for  Simulation 
Science,  Part  of  SPIE  AeroSense  '01  Conference,  Orlando,  Florida,  April  16-18.  pp.  144-151. 

•  Kim,  Taewoo  and  Fishwick,  Paul.  "A  3D-Based  Visualization  Framework  for  Dynamic 
Models",  In  Proceedings  of  the  Web3D  2002  Symposium,  Tempe,  Arizona,  February  2002, 
pp.  103-109. 

•  Fishwick,  Paul.  "Next  Generation  Modeling:  A  Grand  Challenge",  International  Conference 
on  Grand  Challenges  for  Modeling  and  Simulation,  W.H.  Lunceford  and  Ernest  H.  Page, 
Eds,  January  27-31,  2002,  San  Antonio,  pp.  25-30. 

•  Fishwick,  Paul.  '  RUBE:  An  XML-Based  Architecture  for  3D  Process  Modeling  and  Model 
Fusion",  In  Proceedings  of  Enabling  Technology  for  Simulation  Science,  Part  of  SPIE 
Aerosense  ’02  Conference,  Orlando,  Florida,  April  5-9,  2002,  pp.  330-335. 

•  Kim,  Taewoo  and  Fishwick,  Paul.  "An  XML-Based  Visualization  and  Simulation 
Framework  for  Dynamic  Models'Modeling  and  Model  Fusion",  In  Proceedings  of  Enabling 
Technology  for  Simulation  Science,  Part  of  SPIE  Aerosense  '02  Conference,  Orlando, 
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Conference,  Orlando,  Florida,  April  5-9,  2002,  pp.  359-366. 

•  Park,  Minho  and  Fishwick,  Paul.  "SimPackJ/S:  A  Web-Oriented  Toolkit  for  Discrete  Event 
Simulation",  In  Proceedings  of  Enabling  Technology  for  Simulation  Science,  Part  of  SPIE 
Aerosense  '02  Conference,  Orlando,  Florida,  April  5-9,  2002,  pp.  348-358. 
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